Neuromorphic processor and operating method thereof

ABSTRACT

An operating method of a neuromorphic processor which processes data based on a neural network including a first layer including axons and a second layer including neurons includes receiving synaptic weights between the first layer and the second layer, decomposing the synaptic weights into presynaptic weights, a number of which is identical to a number of the axons, and postsynaptic weights, a number of which is identical to a number of the synaptic weights, and storing the presynaptic weights and the postsynaptic weights. A precision of each of the synaptic weights is a first number of bits, a precision of each of the presynaptic weights is a second number of bits, and a precision of each of the postsynaptic weights is a third number of bits. The third number of the bits is smaller than the first number of the bits.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. § 119 to Korean PatentApplication No. 10-2018-0074265 filed on Jun. 27, 2018, in the KoreanIntellectual Property Office, the disclosure of which is herebyincorporated by reference.

BACKGROUND

Embodiments of the inventive concepts described herein relate to aneuromorphic processor, and more particularly, relate to a neuromorphicprocessor to decompose a synaptic weight between an axon and a neuronand an operating method thereof.

The brain contains hundreds of billions of nerve cells, that is,neurons, each of which may learn and remember information whileexchanging signals with another neuron through synapses. When a sum ofsynapse potentials input through dendrites is greater than a thresholdpotential, the neuron may generate an action potential to transfer asignal to another neuron through an axon. This is called “spiking of aneuron”. In the case where the spiking of the neuron occurs, theintensity of the signal transferred from the neuron may vary with theconnection strength of a synapse between a neuron and another neuron.That is, the intensity of the signal that is transferred to the neuronmay vary when the connection strength of the synapse is adjusted, andthus, the learning and memory of information may be made.

The neuromorphic processor is a semiconductor circuit designed to mimica process where a biological nerve cell (or a neuron) transfers andprocesses information. The neuromorphic processor may store a synapticweight being a value corresponding to the connection strength of thesynapse in a memory and may perform signal processing based on thestored weight. That is, the neuromorphic processor may be used toimplement a system that is intellectualized to mimic the brain of thehuman. Accordingly, the neuromorphic processor may be used for characterrecognition, voice recognition, danger recognition, real-time high-speedsignal processing, etc.

However, because there are a large lot of synapses connecting neurons,the neuromorphic processor may require a very great memory usage for thepurpose of storing weight values for the synapses. This may mean thatthe size of the memory included in the neuromorphic processor becomelarge and that the chip area of the neuromorphic processor increases.

SUMMARY

Embodiments of the inventive concepts provide a neuromorphic processorcapable of reducing the size of a memory used to store synaptic weightsand an operating method thereof.

According to an embodiment of the inventive concepts, an operatingmethod of a neuromorphic processor which processes data based on aneural network including a first layer including axons and a secondlayer including neurons includes receiving synaptic weights between thefirst layer and the second layer, decomposing the synaptic weights intopresynaptic weights, a number of which is identical to a number of theaxons, and postsynaptic weights, a number of which is identical to anumber of the synaptic weights, and storing the presynaptic weights andthe postsynaptic weights. A precision of each of the synaptic weights isa first number of bits, a precision of each of the presynaptic weightsis a second number of bits, and a precision of each of the postsynapticweights is a third number of bits. The third number of the bits issmaller than the first number of the bits.

In an embodiment, the synaptic weights may form an M×N weight matrix,the presynaptic weights may form a first sub-weight matrix being an M×1matrix, the postsynaptic weights may form a second sub-weight matrixbeing an M×N matrix, the “M” may be a natural number corresponding tothe number of the axons, and the “N” may be a natural numbercorresponding to the number of the neurons.

In an embodiment, the decomposing may include generating thepostsynaptic weights, and generating a presynaptic weight correspondingto each of the axons based on values of the synaptic weights betweeneach of the axons and the neurons. An absolute value of each of thepostsynaptic weights respectively corresponding to synaptic weightsbeing not “0” from among the synaptic weights is an absolute value orless of a corresponding synaptic weight and a value of each of thepostsynaptic weights respectively corresponding to synaptic weightsbeing “0” from among the synaptic weights is “0”.

In an embodiment, the absolute value of each of the postsynaptic weightsrespectively corresponding to the synaptic weights being not “0” may begenerated to have “1”.

In an embodiment, the presynaptic weight corresponding to each of theaxons may be generated to have a value proportional to a value that isobtained by performing root-mean-square on the values of the synapticweights between each of the axons and the neurons.

The method according to an embodiment of the inventive concepts mayfurther include generating combination weights by multiplying onepresynaptic weight of the presynaptic weights and postsynaptic weightscorresponding to the one presynaptic weight, and outputting a spikebased on the combination weights.

The method according to an embodiment of the inventive concepts mayfurther include updating the decomposed postsynaptic weights based onthe output of the spike.

In an embodiment, an update value for the decomposed postsynapticweights may be determined based on a time to output the spike.

The method according to an embodiment of the inventive concepts mayfurther include updating the decomposed postsynaptic weights based onvalues of the updated postsynaptic weights.

According to an embodiment of the inventive concepts, a neuromorphicprocessor includes a synaptic weight decomposing module that decomposessynaptic weights between a first layer including axons and a secondlayer including neurons into presynaptic weights, a number of which isidentical to a number of the axons, and postsynaptic weights, a numberof which is identical to a number of the synaptic weights, a firstsynapse memory that stores the presynaptic weights, and a second synapsememory that stores the postsynaptic weights. A precision of each of thesynaptic weights is a first number of bits, a precision of each of thepresynaptic weights is a second number of bits, and a precision of eachof the postsynaptic weights is a third number of bits. The third numberof the bits is smaller than the first number of the bits.

In an embodiment, the synaptic weight decomposing module may generatethe postsynaptic weights. An absolute value of each of the postsynapticweights respectively corresponding to synaptic weights being not “0”from among the synaptic weights is an absolute value or less of acorresponding synaptic weight and a value of each of the postsynapticweights respectively corresponding to synaptic weights being “0” fromamong the synaptic weights is “0”.

In an embodiment, the absolute value of each of the postsynaptic weightsrespectively corresponding to the synaptic weights being not “0” is “1”.

In an embodiment, the synaptic weight decomposing module may generate apresynaptic weight corresponding to a specific axon of the axons basedon values of synaptic weights between the specific axon and the neurons.

In an embodiment, the presynaptic weight corresponding to the specificaxon has a value proportional to a value that is obtained by performingroot-mean-square on the values of the synaptic weights between thespecific axon and the neurons.

The neuromorphic processor according to an embodiment of the inventiveconcepts may further include an axon module that outputs a spikecorresponding to a specific axon of the axons, the first synapse memorymay output a presynaptic weight corresponding to the specific axon inresponse to the spike, and the second synapse memory may outputpostsynaptic weights corresponding to the specific axon in response tothe spike.

The neuromorphic processor according to an embodiment of the inventiveconcepts may further include a decomposition weight combining modulethat multiplies the presynaptic weight corresponding to the specificaxon and the postsynaptic weights corresponding to the specific axontogether to generate combination weights.

The neuromorphic processor according to an embodiment of the inventiveconcepts may further include a neuron module that receives thecombination weights and outputs a spike corresponding to each of theneurons based on a combination weight corresponding to each of theneurons.

In an embodiment, the neuron module may update the postsynaptic weightsstored in the second synapse memory based on the output of the spikecorresponding to each of the neurons.

In an embodiment, the neuron module may determine an update value forthe decomposed postsynaptic weights based on a time to output the spike.

In an embodiment, the neuron module may update the presynaptic weightsstored in the first synapse memory based on values of the updatedpostsynaptic weights.

BRIEF DESCRIPTION OF THE FIGURES

The above and other objects and features of the inventive concepts willbecome apparent by describing in detail exemplary embodiments thereofwith reference to the accompanying drawings.

FIG. 1 is a diagram illustrating an example of a neural networkaccording to an embodiment of the inventive concepts.

FIG. 2 is a diagram illustrating a connection state of synapses of aneural network of FIG. 1 in the form of an array.

FIG. 3 is a block diagram illustrating a neuromorphic processoraccording to an embodiment of the inventive concepts.

FIG. 4 is a block diagram illustrating a neuromorphic processor of FIG.3 in detail.

FIG. 5 is a diagram illustrating an example of a neural networkaccording to an embodiment of the inventive concepts.

FIG. 6 is a diagram illustrating an operation of a synaptic weightdecomposing module of FIG. 4 .

FIG. 7 is a diagram illustrating an operation of an axon module of FIG.4 .

FIG. 8 is a diagram illustrating an operation of a decomposition weightcombining module and a neuron module of FIG. 4 .

FIG. 9 is a diagram illustrating an update operation of a neuron moduleof FIG. 4 .

FIG. 10 is a flowchart illustrating a synaptic weight decomposingoperation of a neuromorphic processor of FIG. 3 .

FIG. 11 is a block diagram illustrating a learning operation of aneuromorphic processor of FIG. 3 .

FIG. 12 is a block diagram illustrating an inference operation of aneuromorphic processor of FIG. 3 .

DETAILED DESCRIPTION

Hereinafter, embodiments of the inventive concepts are described indetail with reference to the accompanying drawings. In the followingdescription, specific details such as detailed components and structuresare merely provided to assist the overall understanding of theembodiments of the inventive concepts. Therefore, it should be apparentto those skilled in the art that various changes and modifications ofthe embodiments described herein may be made without departing from thescope and spirit of the present invention. In addition, descriptions ofwell-known functions and structures are omitted for clarity andconciseness. The terms described below are terms defined inconsideration of the functions in the inventive concepts and are notlimited to a specific function. The definitions of the terms should bedetermined based on the contents throughout the specification.

In the following drawings or in the detailed description, modules may beillustrated in a drawing or may be connected with any other componentsother than components in the detailed description. Modules or componentsmay be connected directly or indirectly. Modules or components may beconnected through communication or may be physically connected.

Components that are described in the detailed description with referenceto the terms “unit”, “module”, “˜er or ˜or”, etc. will be implementedwith software, hardware, or a combination thereof. In an embodiment, thesoftware may be a machine code, firmware, an embedded code, andapplication software. For example, the hardware may include anelectrical circuit, an electronic circuit, a processor, a computer, anintegrated circuit, integrated circuit cores, a pressure sensor, amicroelectromechanical system (MEMS), a passive element, or acombination thereof.

Unless defined differently, all terms used herein, which includetechnical terminologies or scientific terminologies, have the samemeaning as that understood by a person skilled in the art to which thepresent invention belongs. Terms defined in a generally used dictionaryare to be interpreted to have meanings equal to the contextual meaningsin a relevant technical field, and are not interpreted to have ideal orexcessively formal meanings unless clearly defined in the specification.

FIG. 1 is a diagram illustrating an example of a neural networkaccording to an embodiment of the inventive concepts. Referring to FIG.1 , a neural network may include axons A1 to Am, neurons N1 to Nn, andsynapses “S”. The synapses “S” may connect the axons A1 to Am and theneurons N1 to Nn. The axons A1 to Am may be included in a first layer L1of the neural network, and the neurons N1 to Nn may be included in asecond layer L2 of the neural network. The first layer L1 may includethe “m” axons A1 to Am, and the second layer L2 may include the “n”neurons N1 to Nn. Here, “m” and “n” may be any natural number, and “m”and “n” may be different numbers or the same number.

Each of the axons A1 to Am of the first layer L1 may output an axonspike. The synapses “S” may transfer signals to the neurons N1 to Nn ofthe second layer L2 based on the output axon spike. Even though an axonspike is output from one axon, each of the signals that are transferredfrom the synapses “S” to the neurons N1 to Nn may vary with a synapticweight being the connection strength of each of the synapses “S”. Forexample, in the case where a synaptic weight of a first synapse isgreater than a synaptic weight of a second synapse, a neuron connectedwith the first synapse may receive a signal of a greater value than aneuron connected with the second synapse.

The neurons N1 to Nn of the second layer L2 may receive signalstransferred from the synapses “S” and may output a neuron spike(s) basedon the received signals. In an embodiment, when a value being a resultof accumulating received signals is greater than a threshold value, eachof the neurons N1 to Nn may output a neuron spike.

For example, as illustrated in FIG. 1 , in the case where the secondaxon A2 outputs an axon spike, the synapses “S” connected with thesecond axon A2 may transfer signals to the neurons N1 to Nn. Thetransferred signals may vary with synaptic weights of the synapses “S”connected with the second axon A2. A signal may be transferred to thesecond neuron N2 from a synapse S22 connecting the second axon A2 andthe second neuron N2; in the case where an accumulated signal value ofthe second neuron N2 becomes greater than the threshold value by thetransferred signal, the second neuron N2 may output a neuron spike.

As illustrated in FIG. 1 , in embodiments of the inventive concepts, alayer where the axons A1 to Am are included may be a layer prior to alayer where the neurons N1 to Nn are included. Also, the layer where theneurons N1 to Nn are included may be a layer following the layer wherethe axons A1 to Am are included. Accordingly, signals may be transferredfrom the neurons N1 to Nn of a next layer based on axon spikes outputfrom the axons A1 to Am, and the neurons N1 to Nn may output neuronspikes based on the transferred signals. That is, the axons A1 to Am mayperform a role of transferring signals to the neurons N1 to Nn of thenext layer, and the neurons N1 to Nn may perform a role of outputtingneuron spikes based on the transferred signals.

Although not illustrated in FIG. 1 , as neuron spikes are output fromthe second layer L2, signals may be transferred to neurons of a nextlayer (i.e., a third layer). In this case, as neuron spikes are output,axons of the second layer L2 may output axon spikes; signals may betransferred to the neurons of the third layer based on the output axonspikes. The neurons of the third layer may output neuron spikes based onthe transferred signals. That is, one layer may include both axons andneurons. However, the inventive concept is not limited thereto. Forexample, one layer may include either axons or neurons.

An example is illustrated in FIG. 1 as the neural network includes thefirst layer L1 and the second layer L2, but the inventive concepts isnot limited thereto. For example, the number of layers included in theneural network may be variously changed. Regardless of the number oflayers, each layer may operate in a manner that is similar to the mannerdescribed with reference to FIG. 1 . Accordingly, the neural network ofFIG. 1 may be expanded to a neural network of various layers.

FIG. 2 is a diagram illustrating a connection state of synapses of aneural network of FIG. 1 in the form of an array. Referring to FIG. 2 ,a neural network may include the axons A1 to Am, the neurons N1 to Nn,and synapses S11 to Smn. The first axon A1 may be connected with theneurons N1 to Nn through the synapses S11 to S1 n, and the second axonA2 may be connected with the neurons N1 to Nn through the synapses S21to S2 n. Likewise, an m-th axon Am may be connected with the neurons N1to Nn through the synapses Sm1 to Smn.

The first axon A1 may be connected with the synapses S11 to S1 n througha first horizontal line PL1, and the second axon A2 may be connectedwith the synapses S21 to S2 n through a second horizontal line PL2.Likewise, the m-th axon Am may be connected with the synapses Sm1 to Smnthrough an m-th horizontal line PLm. The first neuron N1 may beconnected with the synapses S11 to Sm1 through a first vertical lineVL1, and the second neuron N2 may be connected with the synapses S12 toSm2 through a second vertical line VL2. Likewise, the n-th neuron Nn maybe connected with the synapses S1 n to Smn through an n-th vertical lineVLn. One horizontal line may be a signal line through which one axontransfers an axon spike, and one vertical line may be a signal line thatsynapses transfer a synaptic weight-based signal to one neuron inresponse to an axon spike.

Each of the m×n synapses S11 to Smn may include a synaptic weight.Synaptic weights w11 to wmn respectively corresponding to the synapsesS11 to Smn are illustrated in FIG. 2 , and a weight matrix “W” mayinclude the m×n synaptic weights w11 to wmn. That is, the weight matrix“W” may be an m×n matrix.

As illustrated in FIG. 2 , in the case where the second axon A2 outputsan axon spike, the axon spike may be transferred to the synapses S21 toS2 n through the second horizontal line PL2. In response to the receivedaxon spike, the synapses S21 to S2 n may respectively transfer signalsto the neurons N1 to Nn based on synaptic weights of the synapses S21 toS2 n. The signals transferred to the neurons N1 to Nn may be signalsrespectively corresponding to the synaptic weights. For example, thesynapse S22 connecting the second axon A2 and the second neuron N2 maytransfer a signal corresponding to the synaptic weight w22 to the secondneuron N2. In the case where a signal value accumulated according to thetransferred signal is greater than a threshold value stored in advance,the second neuron N2 may output a neuron spike.

In the case of storing the synaptic weights w11 to wmn of FIG. 2 in amemory, the total capacity of the memory necessary to store the synapticweights w11 to wmn may correspond to a value that is obtained bymultiplying the number of synaptic weights and the precision of asynaptic weight together. The precision of a synaptic weight mayindicate a range of a synaptic weight value; as the precision of thesynaptic weight becomes higher, the number of bits for expressing asynaptic weight may increase. That is, the number of bits is the numberof bits indicating a value of each synaptic weight. For example, in thecase where the number of synaptic weights w11 to wmn is “m×n” and theprecision of each synaptic weight corresponds to “p” bits, the capacityof a memory necessary to store the synaptic weights w11 to wmn may be“m×n×p”. In general, because the number of synaptic weights w11 to wmnis great, the capacity of the memory necessary to store the synapticweights w11 to wmn may be considerably great as the precision of thesynaptic weight increases. Accordingly, in the case of storing thesynaptic weights w11 to wmn of FIG. 2 without modification, the usage ofthe memory may become great.

FIG. 3 is a block diagram illustrating a neuromorphic processoraccording to an embodiment of the inventive concepts. Referring to FIG.3 , a neuromorphic processor 100 may receive input data and may processthe received input data. The neuromorphic processor 100 may outputresult data that are generated by processing the input data. Forexample, in the case where the neuromorphic processor 100 is used forcharacter recognition, the input data may include image data, a synapticweight, and topology data of a neural network, and the result data mayinclude data indicating a specific character determined from the imagedata and the synaptic weight. That is, the neuromorphic processor 100may perform learning (e.g. training) based on the input data and mayperform inference for determining the result data with regard to theinput data.

The neuromorphic processor 100 or components of the neuromorphicprocessor 100 may be implemented by using hardware, software, firmware,or a combination thereof. For example, the neuromorphic processor 100 orthe components of the neuromorphic processor 100 may be implemented onone integrated circuit or separate integrated circuits. The neuromorphicprocessor 100 may be mounted on an Internet of Things (IoT) sensor, asmartphone, a desktop computer, a server, etc. Also, the neuromorphicprocessor 100 may be included in a central processing unit (CPU), anapplication processor (AP), a micro controller unit (or microcontroller)(MCU), an accelerator, etc. However, the inventive concepts is notlimited thereto. For example, the neuromorphic processor 100 may be inthe form of a separate chip.

The neuromorphic processor 100 may include a synaptic weight decomposingmodule 110, a first synapse memory 120-1, and a second synapse memory120-2. The synaptic weight decomposing module 110 may decompose synapticweights into presynaptic weights and postsynaptic weights. The synapticweights may be data that are included in the input data and are providedto neuromorphic processor 100. In an embodiment, the precision (i.e.,the number of bits) of each synaptic weight may correspond to “p” bits,the precision (i.e., the number of bits) of each presynaptic weight maycorrespond to “k” bits, and the precision (i.e., the number of bits) ofeach postsynaptic weight may correspond to “q” bits. In this case, theprecision of the postsynaptic weight may be lower than the precision ofthe synaptic weight. That is, “q” may be smaller than “p”, and thepostsynaptic weight may be expressed by using bits, the number of whichis smaller than the number of bits of the synaptic weight.

Alternatively, in the case where the numbers of bits necessary toexpress respective synaptic weights are different, the synaptic weightdecomposing module 110 may decompose synaptic weights such that each ofpostsynaptic weights is expressed by using bits, the number of which issmaller than the greatest number of bits from among the numbers of bitsfor expressing synaptic weights.

As illustrated in FIG. 2 , in the case where the synaptic weights w11 towmn are provided, the synaptic weight decomposing module 110 maydecompose the synaptic weights w11 to wmn into the presynaptic weightsand the postsynaptic weights. The number of presynaptic weightsgenerated as a result of the decomposition may be identical to thenumber of the axons A1 to Am, that is, “m”, and the number ofpostsynaptic weights generated as a result of the decomposition may beidentical to the number of the synaptic weights w11 to wmn, that is,“m×n”. That is, in the case where the m×n weight matrix “W” of FIG. 2 isprovided, the synaptic weight decomposing module 110 may decompose theweight matrix “W” into a first sub-weight matrix being an “m×F” matrixand a second sub-weight matrix being an “m×n” matrix.

The presynaptic weights generated from the synaptic weight decomposingmodule 110 may be stored in the first synapse memory 120-1, and thepostsynaptic weights generated from the synaptic weight decomposingmodule 110 may be stored in the second synapse memory 120-2.

In the case where the synaptic weight decomposing module 110 decomposesthe “m×n” synaptic weights to generate “m” presynaptic weights expressedby “k” bits and “m×n” postsynaptic weights expressed by “q” bits, acapacity of the first synapse memory 120-1 necessary to store thepresynaptic weights may be “m×k”, and a capacity of the second synapsememory 120-2 necessary to store the postsynaptic weights may be “m×n×q”.Accordingly, the total available capacity of the first synapse memory120-1 and the second synapse memory 120-2 may be “(m×k)+(m×n×q)”. Ingeneral, because the number (i.e., “m”) of axons of a neural network andthe number (i.e., “n”) of neurons of the neural network are considerablygreater than numbers (i.e., “p”, “k”, and “q”) each indicating thenumber of bits, the capacity (m×k) of the first synapse memory 120-1 maybe ignorable with regard to the total memory capacity of“(m×k)+(m×n×q)”, and the total memory capacity to be used may be about“m×n×q”.

As described with reference to FIG. 2 , a memory capacity necessary tostore “m×n” synaptic weights expressed by “p” bits may be “m×n×p”.Because “q” being the number of bits of a postsynaptic weight is smallerthan “p” being the number of bits of a synaptic weight, “m×n×q” issmaller than “m×n×p”. Accordingly, a memory capacity that is used aftersynaptic weights are decomposed may be smaller than a memory capacitythat is used before synaptic weights are decomposed. Also, because theprecision of a presynaptic weight compensate a low precision of apostsynaptic weight, even though synaptic weights are decomposed, theaccuracy of the neuromorphic processor 100 may not decrease. That is,the neuromorphic processor 100 according to embodiments of the inventiveconcepts may decompose synaptic weights into presynaptic weights andpostsynaptic weights, thus reducing usage of a memory while maintainingthe accuracy for data operation and processing.

FIG. 4 is a block diagram illustrating a neuromorphic processor of FIG.3 in detail. Referring to FIG. 4 , the neuromorphic processor 100 mayinclude the synaptic weight decomposing module 110, the first synapsememory 120-1, the second synapse memory 120-2, an axon module 130, adecomposition weight combining module 140, and a neuron module 150.Operations of the synaptic weight decomposing module 110, the firstsynapse memory 120-1, and the second synapse memory 120-2 of FIG. 4 aresimilar to the operations of the synaptic weight decomposing module 110,the first synapse memory 120-1, and the second synapse memory 120-2 ofFIG. 3 , and thus, additional description will be omitted to avoidredundancy.

The synaptic weight decomposing module 110 may decompose synapticweights expressed by “p” bits into presynaptic weights expressed by “k”bits and postsynaptic weights expressed by “q” bits. In the case wheresynaptic weights between a first layer including axons and a secondlayer including neurons are decomposed, the number of presynapticweights may be identical to the number of axons, and the number ofpostsynaptic weights may be identical to the number of synaptic weights(i.e., the number of synapses). Also, “q” being the number of bits of apostsynaptic weight may be smaller than “p” being the number of bits ofa synaptic weight.

The first synapse memory 120-1 may store presynaptic weights providedfrom the synaptic weight decomposing module 110. The second synapsememory 120-2 may store postsynaptic weights provided from the synapticweight decomposing module 110.

Each of the first synapse memory 120-1 and the second synapse memory120-2 may be implemented with one of a volatile memory device, such as astatic random access memory (SRAM), a dynamic RAM (DRAM), or asynchronous DRAM (SDRAM), and a nonvolatile memory device, such as aread only memory (ROM), a programmable ROM (PROM), an electricallyprogrammable ROM (EPROM), an electrically erasable and programmable ROM(EEPROM), a phase-change RAM (PRAM), a magnetic RAM (MRAM), a resistiveRAM (RRAM), a ferroelectric RAM (FRAM), or a flash memory. Asillustrated in FIG. 4 , the first synapse memory 120-1 and the secondsynapse memory 120-2 may be included in one synapse memory 120, but theinventive concepts is not limited thereto. For example, the firstsynapse memory 120-1 and the second synapse memory 120-2 may beimplemented with separate memories, respectively.

The axon module 130 may output an axon spike corresponding to a specificaxon from among axons to the synapse memory 120. In response to the axonspike, the synapse memory 120 may output a presynaptic weightcorresponding to the specific axon from the first synapse memory 120-1and may output postsynaptic weights corresponding to the specific axonfrom the second synapse memory 120-2. Presynaptic weights stored in thefirst synapse memory 120-1 may respectively correspond to axons, andthus, one presynaptic weight corresponding to the specific axon may beoutput.

Postsynaptic weights stored in the second synapse memory 120-2 mayrespectively correspond to synapses connecting axons and neurons. Thismeans that postsynaptic weights corresponding to the specific axon existas much as the number of synapses connecting the specific axon and theneurons. For example, in the case where “m×n” postsynaptic weights arestored in the second synapse memory 120-2, the number of postsynapticweights corresponding to the specific axon may be “n”.

The axon module 130 may receive a neuron spike output from the neuronmodule 150. The axon module 130 may output an axon spike (or axonspikes) corresponding to at least one (or two axons) of axons inresponse to the neuron spike. For example, in the case where the numberof neurons is identical to the number of axons and a neuron spike isoutput from a specific neuron, the axon module 130 may determine an axoncorresponding to the specific neuron and may output an axon spikecorresponding to the determined axon. That is, the axon module 130 mayin advance store correspondence information between one neuron and acorresponding axon and may output an axon spike based on the storedcorrespondence information when a neuron spike is received.

The decomposition weight combining module 140 may receive a presynapticweight output from the first synapse memory 120-1 and postsynapticweights output from the second synapse memory 120-2. The decompositionweight combining module 140 may generate combination weights bymultiplying the presynaptic weight and the postsynaptic weightstogether. A combination weight may be a value that is obtained bymultiplying a presynaptic weight and a relevant postsynaptic weighttogether. This means that the number of combination weights is identicalto the number of postsynaptic weights and the combination weightscorrespond to neurons, respectively.

Because a combination weight is a value obtained by multiplying apresynaptic weight and a postsynaptic weight, the number of bits of acombination weight may be greater than the number of bits of apresynaptic weight or the number of bits of a postsynaptic weight. Forexample, in the case where the number of bits of a presynaptic weight is“k” and the number of bits of a postsynaptic weight is “q”, the numberof bits of a combination weight may be “(k+q)” bits. The generatedcombination weights may be provided to the neuron module 150, and thus,the neuron module 150 may perform an operation based on a combinationweight of high precision. Because the neuron module 150 operates basedon a combination weight of high precision, the neuromorphic processor100 may maintain precision in operating and processing data.

The neuron module 150 may receive combination weights. The neuron module150 may output a neuron spike corresponding to each neuron based on acombination weight corresponding to each neuron. For example, the neuronmodule 150 may increase or decrease a signal value corresponding to aspecific neuron based on a combination weight. In the case where asignal value is greater than a threshold value stored in advance, theneuron module 150 may output a neuron spike corresponding to thespecific neuron. The neuron spike output from the neuron module 150 maybe provided to the axon module 130.

The neuron module 150 may output an update signal based on whether aneuron spike is output. The second synapse memory 120-2 may updatevalues of stored postsynaptic weights in response to the update signal.For example, in the case where a neuron spike corresponding to aspecific neuron is output, the neuron module 150 may output the updatesignal for updating weights corresponding to synapses connected to thespecific neuron. The second synapse memory 120-2 may update postsynapticweights corresponding to the synapses connected to the specific neuronin response to the update signal. In the case where “m×n” postsynapticweights are stored, the second synapse memory 120-2 may update “m”postsynaptic weights corresponding to the specific neuron.

Alternatively, in response to the update signal of the neuron module150, the first synapse memory 120-1 may update presynaptic weights.Specifically, the neuron module 150 may determine the number of weightsgreater than a given value from among updated postsynaptic weights. Inthe case where the number of postsynaptic weights greater than the givenvalue is greater than a given number, the neuron module 150 may updatepresynaptic weights and postsynaptic weights. Here, the given value mayvary with the precision (i.e., the number of bits) of a postsynapticweight, and the given number may vary with the total number ofpostsynaptic weights (i.e., the total number of synaptic weights).Accordingly, the number of times that presynaptic weights are updatedmay be less than the number of times that postsynaptic weights areupdated.

In the case where the neuromorphic processor 100 performs learning oninput data, postsynaptic weights may be updated according to the updatesignal from the neuron module 150. In the case where only postsynapticweights are updated in the learning operation, the total number of timesof an access to the synapse memory 120 may decrease, and thus, an updateoperation may be quickly performed.

In the case where the neuromorphic processor 100 performs inference oninput data, the neuromorphic processor 100 may not output the updatesignal. The neuromorphic processor 100 may output result data based on aneuron spike output from the neuron module 150.

FIG. 5 is a diagram illustrating an example of a neural networkaccording to an embodiment of the inventive concepts. Referring to FIG.5 , a neural network may include the first and second axons A1 and A2,the first to third neurons N1 to N3, and synapses S11 to S23. The firstaxon A1 may be connected with the first to third neurons N1 to N3through the synapses S11 to S13, and the second axon A2 may be connectedwith the first to third neurons N1 to N3 through the synapses S21 toS23. Synaptic weights w11 to w23 of the weight matrix “W” may indicatethe connection strengths of the synapses S11 to S23. That is, the neuralnetwork of FIG. 5 may include “2×3” synaptic weights w11 to w23.

Below, an operation of the neuromorphic processor 100 of FIG. 4 will bemore fully described with reference to the neural network of FIG. 5 .However, the inventive concepts is not limited to the neural network ofFIG. 5 . For example, the neuromorphic processor 100 of the inventiveconcepts may be applied to various neural networks.

FIG. 6 is a diagram illustrating an operation of a synaptic weightdecomposing module of FIG. 4 . The synaptic weight decomposing module110 may decompose six synaptic weights w11 to w23 into two presynapticweights prw1 and prw2 and six postsynaptic weights pow11 to pow23. Thatis, the synaptic weight decomposing module 110 may decompose the “2×3”weight matrix “W” into a first sub-weight matrix SW1 being a “2×1”matrix and a second sub-weight matrix SW2 being a “2×3” matrix.

Each of the synaptic weights w11 to w23 may be expressed by “p” bits,each of the presynaptic weights prw1 and prw2 may be expressed by “k”bits, and each of the postsynaptic weights pow11 to pow23 may beexpressed by “q” bits. Here, “q” being the number of bits of apostsynaptic weight may be smaller than “p” being the number of bits ofa synaptic weight.

The synaptic weight decomposing module 110 may generate the presynapticweights prw1 and prw2, the number of which is identical to the number ofaxons A1 and A2 of FIG. 5 . As such, the presynaptic weight prw1 maycorrespond to the first axon A1, and the presynaptic weight prw2 maycorrespond to the second axon A2.

The synaptic weight decomposing module 110 may generate the postsynapticweights pow11 to pow23, the number (i.e., 6) of which is identical tothe number of synaptic weights w11 to w23 or synapses S11 and S23 ofFIG. 5 . The postsynaptic weights pow11 to pow13 at the first row of thesecond sub-weight matrix SW2 may correspond to the first axon A1, andthe postsynaptic weights pow21 to pow23 at the second row of the secondsub-weight matrix SW2 may correspond to the second axon A2.

For example, in the case where the weight matrix “W” is as follow,

$W = \begin{bmatrix}0 & 1 & 2 \\2 & 4 & 0\end{bmatrix}$

the synaptic weight decomposing module 110 may decompose the weightmatrix “W” illustrated above the synaptic weight decomposing module 110into the first sub-weight matrix SW1 and the second sub-weight matrixSW2 illustrated below the synaptic weight decomposing module 110.

${{SW}\; 1} = \begin{bmatrix}1 \\3\end{bmatrix}$ ${{SW}\; 2} = \begin{bmatrix}0 & 1 & 1 \\1 & 1 & 0\end{bmatrix}$

In this case, each synaptic weight may be expressed by three bits, eachpresynaptic weight may be expressed by two bits, and each postsynapticweight may be expressed by one bit. The number (i.e., 1) of bits of apostsynaptic weight may be smaller than the number (i.e., 3) of bits ofa synaptic weight.

The synaptic weight decomposing module 110 may generate an absolutevalue of each of postsynaptic weights respectively corresponding tosynaptic weights being not “0” from among synaptic weights, so as tohave an absolute value or less of a corresponding synaptic weight andmay generate a value of each of postsynaptic weights respectivelycorresponding to synaptic weights being “0” from among the synapticweights so as to have “0”. For example, the synaptic weight decomposingmodule 110 may generate an absolute value of each of postsynapticweights respectively corresponding to synaptic weights being not “0”, soas to have “1”.

The synaptic weight decomposing module 110 may generate a presynapticweight corresponding to one axon based on values of synaptic weightsbetween one axon and neurons. For example, the synaptic weightdecomposing module 110 may generate the presynaptic weight prw1corresponding to the first axon A1 so as to be proportional to a valueobtained by performing a root-mean-square operation on values (i.e., 0,1, and 2) of the synaptic weights w11, w12, and w13 between the firstaxon A1 and the neurons N1 to N3. However, the inventive concepts is notlimited thereto. For example, the synaptic weight decomposing module 110may generate a presynaptic weight based on various methods.

The above weight matrix “W” includes synaptic weights of positivevalues, but the inventive concepts is not limited thereto. For example,the weight matrix “W” may include synaptic weights of negative values.In this case, a presynaptic weight of a negative value and apostsynaptic weight of a negative value may be generated.

FIG. 7 is a diagram illustrating an operation of an axon module of FIG.4 . Referring to FIG. 7 , the axon module 130 may include a firstsub-axon module 131 and a second sub-axon module 132. The first sub-axonmodule 131 may correspond to the first axon A1 of FIG. 5 , and thesecond sub-axon module 132 may correspond to the second axon A2 of FIG.5 . The first sub-axon module 131 may output a first axon spike, and thesecond sub-axon module 132 may output a second axon spike. An axon spikeoutput from the axon module 130 may be provided to the synapse memory120.

The synapse memory 120 may include the first synapse memory 120-1 andthe second synapse memory 120-2. The first synapse memory 120-1 mayinclude a first presynaptic memory block and a second presynaptic memoryblock. Each of the presynaptic memory blocks may include one or morememory cells. The first presynaptic memory block may store thepresynaptic weight prw1, and the second presynaptic memory block maystore the presynaptic weight prw2. The presynaptic weights prw1 and prw2may be provided from the synaptic weight decomposing module 110 of FIG.6 . The presynaptic weight prw1 stored in the first presynaptic memoryblock may correspond to the first axon A1 of FIG. 5 , and thepresynaptic weight prw2 stored in the second presynaptic memory blockmay correspond to the second axon A2 of FIG. 5 .

The second synapse memory 120-2 may include first to sixth postsynapticmemory blocks. Each of the postsynaptic memory blocks may include one ormore memory cells. The first postsynaptic memory block may store thepostsynaptic weight pow11 Likewise, the second to sixth postsynapticmemory blocks may store corresponding postsynaptic weights pow12 topow23, respectively. The postsynaptic weights pow11 to pow13 stored inthe first to third postsynaptic memory blocks may correspond to thefirst axon A1 of FIG. 5 , and the postsynaptic weights pow21 to pow23stored in the fourth to sixth postsynaptic memory blocks may correspondto the second axon A2 of FIG. 5 .

The first synapse memory 120-1 may output a presynaptic weight inresponse to an axon spike provided from the axon module 130. Forexample, the first synapse memory 120-1 may output the presynapticweight prw1 stored in the first presynaptic memory block in response tothe first axon spike. Alternatively, the first synapse memory 120-1 mayoutput the presynaptic weight prw2 stored in the second presynapticmemory block in response to the second axon spike.

The second synapse memory 120-2 may output relevant postsynaptic weightsin response to an axon spike provided from the axon module 130. Forexample, the second synapse memory 120-2 may output the postsynapticweights pow11 to pow13 stored in the first to third postsynaptic memoryblocks in response to the first axon spike. Alternatively, the secondsynapse memory 120-2 may output the postsynaptic weights pow21 to pow23stored in the fourth to sixth postsynaptic memory blocks in response tothe second axon spike.

As described above, the first synapse memory 120-1 and the secondsynapse memory 120-2 may output a presynaptic weight and postsynapticweights corresponding to a specific axon (i.e., a specific axon module)from among axons in response to an axon spike output from the specificaxon.

FIG. 8 is a diagram illustrating an operation of a decomposition weightcombining module and a neuron module of FIG. 4 . Referring to FIG. 8 ,the decomposition weight combining module 140 may receive a presynapticweight and postsynaptic weights from the synapse memory 120 of FIG. 7 .For example, in the case where the synapse memory 120 outputs thepresynaptic weight prw1 and the postsynaptic weights pow11, pow12, andpow13 in response to a first axon spike, the decomposition weightcombining module 140 may receive the presynaptic weight prw1 and thepostsynaptic weights pow11, pow12, and pow13. However, the inventiveconcept is not limited thereto. For example, the decomposition weightcombining module 140 may receive the presynaptic weight prw2 and thepostsynaptic weights pow21, pow22, and pow23. For convenience ofdescription, below, an operation of the decomposition weight combiningmodule 140 will be described with reference to an example in which thepresynaptic weight prw1 and the postsynaptic weights pow11, pow12, andpow13.

The decomposition weight combining module 140 may include a firstmultiplier 141, a second multiplier 142, and a third multiplier 143. Thedecomposition weight combining module 140 may generate combinationweights cw1, cw2, and cw3 by using the first to third multipliers 141 to143. The first multiplier 141 may generate the combination weight cw1 bymultiplying the presynaptic weight prw1 and the postsynaptic weightpow11. The second multiplier 142 may generate the combination weight cw2by multiplying the presynaptic weight prw1 and the postsynaptic weightpow12. The third multiplier 143 may generate the combination weight cw3by multiplying the presynaptic weight prw1 and the postsynaptic weightpow13. For example, in the case where the presynaptic weight prw1 is “3”and the postsynaptic weights pow11, pow12, and pow13 are “1”, “1”, and“0”, respectively, the combination weights cw1, cw2, and cw3 may be “3”,“3”, and “0”. The decomposition weight combining module 140 may providethe generated combination weights cw1, cw2, and cw3 to the neuron module150.

The combination weights cw1, cw2, and cw3 that are generated based onthe presynaptic weight prw1 and the postsynaptic weights pow11, pow12,and pow13 may correspond to the first axon A1 of FIG. 5 . Thecombination weight cw1 may correspond to the synaptic weight w11 betweenthe first axon A1 and the first neuron N1 of FIG. 5 . The combinationweight cw2 may correspond to the synaptic weight w12 between the firstaxon A1 and the second neuron N2 of FIG. 5 . The combination weight cw3may correspond to the synaptic weight w13 between the first axon A1 andthe third neuron N3 of FIG. 5 . Values of the combination weights cw1,cw2, and cw3 may be identical to or different from values of thecorresponding synaptic weights w11, w12, and w13. That is, a value of agenerated combination weight may be different from a correspondingsynaptic weight, and a difference of the value may be within a givenrange. As such, even though the neuromorphic processor 100 operatesbased on a combination weight, the operation of the neuromorphicprocessor 100 may be similar to an operation of the neuromorphicprocessor 100 based on a synaptic weight.

The neuron module 150 may output a neuron spike based on the combinationweights cw1, cw2, and cw3. The neuron module 150 may include a firstsub-neuron module 151, a second sub-neuron module 152, and a thirdsub-neuron module 153. The first to third sub-neuron modules 151 to 153may correspond to the first to third neurons N1 to N3 of FIG. 5 ,respectively.

The first to third sub-neuron modules 151 to 153 may receive thecorresponding combination weights cw1, cw2, and cw3 and may output aneuron spike based on the combination weights cw1, cw2, and cw3. Forexample, the first sub-neuron module 151 may output a neuron spike basedon the combination weight cw1 Likewise, the second sub-neuron module 152and the third sub-neuron module 153 may output a second neuron spike anda third neuron spike, respectively. A neuron spike output from theneuron module 150 may include at least one of the first to third neuronspikes.

Each of the first to third sub-neuron modules 151 to 153 may receive thecorresponding combination weight and may accumulate and store a signalvalue corresponding to the combination weight. The first to thirdsub-neuron modules 151 to 153 may sequentially receive the correspondingcombination weights and may accumulate and store signal valuesrespectively corresponding to the combination weights. For example,before the first sub-neuron module 151 receives the combination weightcw1, the first sub-neuron module 151 may include an accumulated signalvalue corresponding to the received combination weights. In the casewhere the first sub-neuron module 151 receives the combination weightcw1, the first sub-neuron module 151 may add a signal valuecorresponding to the combination weight cw1 to an existing accumulatedsignal value.

In the case where a signal value accumulated according to the receivedcombination weight is greater than a threshold value, each of the firstto third sub-neuron modules 151 to 153 may output a neuron spike. Forexample, the first sub-neuron module 151 may accumulate a signal valuecorresponding to the received combination weight cw1; when theaccumulated signal value is greater than the threshold value, the firstsub-neuron module 151 may output the first neuron spike. After the firstsub-neuron module 151 outputs the first neuron spike, the firstsub-neuron module 151 may initialize the accumulated signal value.

As described above, the neuron module 150 may output a neuron spikebased on combination weights. The learning or inference operation of theneuromorphic processor 100 may be performed based on a neuron spikeoutput from the neuron module 150. For example, the neuromorphicprocessor 100 may output a determination result of character recognitionas result data, based on the output neuron spike.

FIG. 9 is a diagram illustrating an update operation of a neuron moduleof FIG. 4 . Referring to FIG. 9 , the synapse memory 120 may include thefirst synapse memory 120-1 and the second synapse memory 120-2. Thefirst synapse memory 120-1 may include the first and second presynapticmemory blocks, and the second synapse memory 120-2 may include the firstto sixth postsynaptic memory blocks. The presynaptic weights prw1 andprw2 may be respectively stored in the corresponding presynaptic memoryblocks, and the postsynaptic weights pow11 to pow23 may be respectivelystored in the corresponding postsynaptic memory blocks.

The neuron module 150 may include the first to third sub-neuron modules151 to 153. Each of the first to third sub-neuron modules 151 to 153 mayoutput a neuron spike based on the corresponding combination weight, asdescribed with reference to FIG. 8 . For example, the first sub-neuronmodule 151 may output a first neuron spike based on the combinationweight cw1.

In the learning operation, the neuron module 150 may provide the updatesignal to the synapse memory 120 when a neuron spike is output. Thesynapse memory 120 may update values of stored weights in response tothe update signal. In an embodiment, in response to the update signal,the second synapse memory 120-2 may update stored postsynaptic weights.In the case where a presynaptic weight is not updated and onlypostsynaptic weights are updated, an operation of updating a weight maybe quickly performed. However, the inventive concepts is not limitedthereto. For example, the synapse memory 120 may update a presynapticweight stored in the first synapse memory 120-1.

For example, the neuron module 150 may determine the number of weightsgreater than a given value from among updated postsynaptic weights. Inthe case where the number of postsynaptic weights greater than the givenvalue is greater than a given number, the neuron module 150 may updatepresynaptic weights and postsynaptic weights. Here, the given value mayvary with the precision (i.e., the number of bits) of a postsynapticweight, and the given number may vary with the total number ofpostsynaptic weights (i.e., the total number of synaptic weights).Accordingly, the number of times that presynaptic weights are updatedmay be less than the number of times that postsynaptic weights areupdated.

For example, the neuron module 150 may output the update signal when afirst neuron spike is output. The update signal according to the outputof the first neuron spike may be an update signal associated with thepostsynaptic weights pow11 and pow21 stored in the first and fourthpostsynaptic memory blocks. Likewise, the neuron module 150 may outputan update signal associated with the postsynaptic weights pow12 andpow22 stored in the second and fifth postsynaptic memory blocks when asecond neuron spike is output, and may output an update signalassociated with the postsynaptic weights pow13 and pow23 stored in thethird and sixth postsynaptic memory blocks when a third neuron spike isoutput. That is, when a neuron spike corresponding to a specific neuronis output, postsynaptic weights corresponding to the specific neuron maybe updated.

In an embodiment, the neuron module 150 may provide an address where aweight to be updated is stored and a value of a weight to be updated tothe synapse memory 120. The value of the weight to be updated may bedetermined according to spike timing dependent plasticity (STDP). TheSTDP is an algorithm to increase or decrease a value of a weight inconsideration of a time when a neuron outputs a spike. That is, theneuron module 150 may determine a value of a weight to be updated inconsideration of a time of a spike output from the first to thirdsub-neuron modules 151 to 153.

FIG. 10 is a flowchart illustrating a synaptic weight decomposingoperation of a neuromorphic processor of FIG. 3 . Referring to FIGS. 3and 10 , in operation S101, the neuromorphic processor 100 may receivesynaptic weights expressed by the first number of bits. Synaptic weightsmay indicate connection strengths of synapses between a first layerincluding axons and a second layer including neurons.

In operation S102, the neuromorphic processor 100 may decompose synapticweights into presynaptic weights expressed by the second number of bitsand postsynaptic weights expressed by the third number of bits. Thethird number of bits indicating the precision of a postsynaptic weightmay be smaller than the first number of bits indicating the precision ofa synaptic weight. The number of presynaptic weights that are generatedas a result of the decomposition may be identical to the number of axonsin the first layer. The number of postsynaptic weights that aregenerated as the decomposition result may be identical to the number ofsynaptic weights (i.e., synapses between the first layer and the secondlayer).

In operation S103, the neuromorphic processor 100 may store thepresynaptic weight and the postsynaptic weights generated of thedecomposition result. The neuromorphic processor 100 may store thepresynaptic weights and the postsynaptic weights in an internal memory.Alternatively, the neuromorphic processor 100 may store the presynapticweights and the postsynaptic weights in a memory outside theneuromorphic processor 100.

In an embodiment, in operation S102, the neuromorphic processor 100 maygenerate an absolute value of each of postsynaptic weights correspondingto synaptic weights being not “0” from among synaptic weights, so as tohave an absolute value or less of the corresponding synaptic weight. Forexample, the neuromorphic processor 100 may generate an absolute valueof each of postsynaptic weights corresponding to synaptic weights beingnot “0”, so as to have “1”. The neuromorphic processor 100 may generatea value of each of postsynaptic weights corresponding to synapticweights being “0” from among synaptic weights, so as to have “0”. Theneuromorphic processor 100 may generate presynaptic weights respectivelycorresponding to the axons of the first layer based on synaptic weightsbetween the axons of the first layer and the neurons of the secondlayer.

FIG. 11 is a block diagram illustrating a learning operation of aneuromorphic processor of FIG. 3 . Referring to FIGS. 3 and 11 , inoperation S111, the neuromorphic processor 100 may receive synapticweights expressed by the first number of bits; in operation S112, theneuromorphic processor 100 may decompose the synaptic weights intopresynaptic weights expressed by the second number of bits andpostsynaptic weights expressed by the third number of bits. OperationS111 and operation S112 of FIG. 11 are similar to operation S101 andoperation S102 of FIG. 10 , and thus, additional description will beomitted to avoid redundancy.

In operation S113, the neuromorphic processor 100 may generatecombination weights by multiplying one of presynaptic weights andpostsynaptic weights corresponding to the one presynaptic weighttogether. The generated combination weights may correspond to a specificaxon of axons in a first layer. In operation S114, the neuromorphicprocessor 100 may output a spike based on the generated combinationweights. The neuromorphic processor 100 may output a spike correspondingto each neuron based on a combination weight corresponding to eachneuron. In operation S115, the neuromorphic processor 100 may updatepostsynaptic weights based on the output of the spike. The neuromorphicprocessor 100 may determine a value of a postsynaptic weight to beupdated, based on a time to output a spike corresponding to each neuron.

As illustrated in FIG. 11 , the neuromorphic processor 100 may performoperation S111 to operation S115 to perform learning, but the inventiveconcepts is not limited thereto. For example, the neuromorphic processor100 may perform the learning by continuously updating postsynapticweights through the iteration of operation S113 to operation S115.

FIG. 12 is a block diagram illustrating an inference operation of aneuromorphic processor of FIG. 3 . Referring to FIGS. 3 and 12 , inoperation S121, the neuromorphic processor 100 may receive synapticweights expressed by the first number of bits. In operation S122, theneuromorphic processor 100 may decompose synaptic weights intopresynaptic weights expressed by the second number of bits andpostsynaptic weights expressed by the third number of bits. In operationS123, the neuromorphic processor 100 may generate combination weights bymultiplying one of presynaptic weights and postsynaptic weightscorresponding to the one presynaptic weight together. In operation S124,the neuromorphic processor 100 may output a spike based on thecombination weights. Operation S121 and operation S122 of FIG. 12 aresimilar to operation S101 and operation S102 of FIG. 10 , and thus,additional description will be omitted to avoid redundancy. OperationS123 and operation S124 of FIG. 12 are similar to operation S113 andoperation S114 of FIG. 11 , and thus, additional description will beomitted to avoid redundancy.

In operation S125, the neuromorphic processor 100 may output aninference result based on the output of the spike. For example, in thecase where the neuromorphic processor 100 performs characterrecognition, the neuromorphic processor 100 may receive image data asinput data and may output a character, which the received image dataindicate, as an inference result. In the case where the neurons of thesecond layer correspond to different characters, respectively, theneuromorphic processor 100 may output a character based on a spikeoutput from a specific neuron of the neurons. That is, the neuromorphicprocessor 100 may output a character corresponding to a specific neuronoutputting a spike, as an inference result.

As described above, the neuromorphic processor 100 according toembodiments of the inventive concepts may determine the numbers of bitsof presynaptic weights and postsynaptic weights decomposed from synapticweights such that a memory usage is reduced. The neuromorphic processor100 may perform learning and inference based on combination weightsgenerated from the presynaptic weights and the postsynaptic weights. Assuch, the memory usage of the neuromorphic processor 100 may be reduced,and the accuracy for data operation and processing may be maintained.

According to an embodiment of the inventive concepts, the memory usageof a neuromorphic processor may be reduced.

Also, the neuromorphic processor according to an embodiment of theinventive concepts may maintain the accuracy for data operation andprocessing while using a low-capacity memory.

While the inventive concepts has been described with reference toexemplary embodiments thereof, it will be apparent to those of ordinaryskill in the art that various changes and modifications may be madethereto without departing from the spirit and scope of the inventiveconcepts as set forth in the following claims.

What is claimed is:
 1. An operating method of a neuromorphic processorwhich processes data based on a neural network including a first layerincluding axons and a second layer including neurons, the methodcomprising: by one or more processors of the neuromorphic processor:receiving synaptic weights between the first layer and the second layer;decomposing the synaptic weights into presynaptic weights andpostsynaptic weights, a number of the presynaptic weights is identicalto a number of the axons, and a number of the postsynaptic weights isidentical to a number of the synaptic weights; and storing, in a synapsememory storage, the presynaptic weights and the postsynaptic weights,wherein a precision of each of the synaptic weights is a first number ofbits, a precision of each of the presynaptic weights is a second numberof bits, and a precision of each of the postsynaptic weights is a thirdnumber of bits, and wherein the third number of the bits is smaller thanthe first number of the bits.
 2. The method of claim 1, wherein thesynaptic weights form an M×N weight matrix, the presynaptic weights forma first sub-weight matrix being an M×1 matrix, the postsynaptic weightsform a second sub-weight matrix being an M×N matrix, the “M” is anatural number corresponding to the number of the axons, and the “N” isa natural number corresponding to the number of the neurons.
 3. Themethod of claim 1, wherein the decomposing includes: generating thepostsynaptic weights; and generating, based on values of the synapticweights between each of the axons and the neurons, a presynaptic weightcorresponding to each of the axons, wherein an absolute value of each ofthe postsynaptic weights respectively corresponding to synaptic weightsbeing not “0” from among the synaptic weights is an absolute value orless of a corresponding synaptic weight and a value of each of thepostsynaptic weights respectively corresponding to synaptic weightsbeing “0” from among the synaptic weights is “0”.
 4. The method of claim3, wherein the absolute value of each of the postsynaptic weightsrespectively corresponding to the synaptic weights being not “0” isgenerated to have “1”.
 5. The method of claim 3, wherein the presynapticweight corresponding to each of the axons is generated to have a valueproportional to a value that is obtained by performing root-mean-squareon the values of the synaptic weights between each of the axons and theneurons.
 6. The method of claim 1, further comprising: generatingcombination weights by multiplying one presynaptic weight of thepresynaptic weights and postsynaptic weights corresponding to the onepresynaptic weight; and outputting a spike based on the combinationweights.
 7. The method of claim 6, further comprising: updating thedecomposed postsynaptic weights based on the output of the spike.
 8. Themethod of claim 7, wherein the updating of the decomposed postsynapticweights is determined based on a time to output the spike.
 9. The methodof claim 7, further comprising: updating the decomposed postsynapticweights based on values of the updated postsynaptic weights.
 10. Aneuromorphic processor comprising: a synaptic weight decomposing circuitconfigured to decompose synaptic weights, of synapse connections betweena first layer including axons and a second layer including neurons, intopresynaptic weights and postsynaptic weights, wherein a number of thepresynaptic weights is identical to a number of the axons, and wherein anumber of the postsynaptic weights is identical to a number of thesynaptic weights; a first synapse memory configured to store thepresynaptic weights; and a second synapse memory configured to store thepostsynaptic weights, wherein a precision of each of the synapticweights is a first number of bits, a precision of each of thepresynaptic weights is a second number of bits, and a precision of eachof the postsynaptic weights is a third number of bits, and wherein thethird number of the bits is smaller than the first number of the bits.11. The neuromorphic processor of claim 10, wherein the synaptic weightdecomposing circuit is configured to: generate the postsynaptic weights,wherein an absolute value of each of the postsynaptic weightsrespectively corresponding to synaptic weights being not “0” from amongthe synaptic weights is an absolute value or less of a correspondingsynaptic weight and a value of each of the postsynaptic weightsrespectively corresponding to synaptic weights being “0” from among thesynaptic weights is “0”.
 12. The neuromorphic processor of claim 11,wherein the absolute value of each of the postsynaptic weightsrespectively corresponding to the synaptic weights being not “0” is “1”.13. The neuromorphic processor of claim 10, wherein the synaptic weightdecomposing circuit is configured to: generate, based on values ofsynaptic weights between the specific axon and the neurons, apresynaptic weight corresponding to a specific axon.
 14. Theneuromorphic processor of claim 13, wherein the presynaptic weightcorresponding to the specific axon has a value proportional to a valuethat is obtained by performing root-mean-square on the values of thesynaptic weights between the specific axon and the neurons.
 15. Theneuromorphic processor of claim 10, further comprising: an axon circuitconfigured to output a spike corresponding to a specific axon of theaxons, wherein the first synapse memory is configured to output apresynaptic weight corresponding to the specific axon in response to thespike, and wherein the second synapse memory is configured to outputpostsynaptic weights corresponding to the specific axon in response tothe spike.
 16. The neuromorphic processor of claim 15, furthercomprising: a decomposition weight combining circuit configured tomultiply the presynaptic weight corresponding to the specific axon andthe postsynaptic weights corresponding to the specific axon together togenerate combination weights.
 17. The neuromorphic processor of claim16, further comprising: a neuron circuit configured to receive thecombination weights and to output a spike corresponding to each of theneurons based on a combination weight corresponding to each of theneurons.
 18. The neuromorphic processor of claim 17, wherein the neuroncircuit is further configured to: update the postsynaptic weights storedin the second synapse memory based on the output of the spikecorresponding to each of the neurons.
 19. The neuromorphic processor ofclaim 18, wherein the neuron circuit is configured to: determine anupdate value for the decomposed postsynaptic weights based on a time tooutput the spike.
 20. The neuromorphic processor of claim 18, whereinthe neuron circuit is further configured to: update the presynapticweights stored in the first synapse memory based on values of theupdated postsynaptic weights.